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Abstract 

We  study  heavy  traffic  asymptotics  of  many  Additive  Increase  Multi¬ 
plicative  Decrease  (AIMD)  connections  sharing  a  common  router  in  the 
presence  of  other  uncontrolled  traffic,  called  ’’mice”.  The  system  is  scaled 
by  speed  and  average  number  of  sources.  With  appropriate  scalings  of  the 
packet  rate  and  buffer  content,  an  approximating  delayed  diffusion  model 
is  derived.  By  heavy  traffic  we  mean  that  there  is  relatively  little  spare  ca¬ 
pacity  in  the  operating  regime.  In  contrast  to  previous  scaled  models,  the 
randomness  due  to  the  mice  or  number  of  connections  is  not  averaged, 
and  plays  its  natural  and  dominant  role.  The  asymptotic  heavy  traffic 
model  allows  us  to  analyze  buffer  and  loss  management  policies  of  early 
marking  or  discarding  as  a  function  of  the  queue  size  and/or  the  total 
input  rate  and  to  choose  a  nearly  optimal  function  via  use  of  an  appro¬ 
priate  limiting  optimal  control  problem,  captures  the  essential  features  of 
the  physical  problem,  and  can  guide  us  to  good  operating  policies.  After 
studying  the  asymptotics  of  a  large  number  of  persistent  AIMD  connec¬ 
tions  we  also  handle  the  asymptotics  of  finite  AIMD  connections  whose 
number  varies  as  connections  arrive  and  leave.  The  data  illustrate  some 
of  the  advantages  of  the  approach. 

Keywords:  AIMD  models,  FTP  analysis,  heavy  traffic  analysis,  approximating 
delay-diffusions,  nearly  optimal  controls 


1  Introduction 

Background  and  motivation  One  of  the  most  active  research  areas  in  net¬ 
working  in  recent  years  has  been  the  modeling  and  analysis  of  AIMD  traffic;  e.g., 
[1,  2,  3,  4,  6,  10,  15,  16,  17,  18,  20,  21].  When  considering  a  single  connection 
and  modeling  all  other  connections  through  an  idealized  loss  process,  simple 
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mathematical  formulas  for  the  connection’s  throughput  can  be  obtained;  e.g., 
[1,  6,  18,  21].  However,  it  is  important  in  practice  to  understand  the  interaction 
of  competing  random  numbers  of  connections  and  the  associated  system  ran¬ 
domness  that  determines  both  the  throughput  as  well  as  the  losses  suffered  by 
the  various  connections.  One  approach  is  through  a  fixed  point  argument;  see, 
e.g.,  [4].  If  the  loss  rates  over  the  nodes  (or  links)  traversed  by  the  connections 
are  sufficiently  small  and  can  be  assumed  to  be  additive,  an  alternative  frame¬ 
work  can  be  used  where  the  throughputs  of  TCP  are  obtained  as  the  solution 
of  a  convex  optimization  problem  and  where  the  loss  probabilities  are  obtained 
as  the  Lagrange  multipliers  [16,  21]. 

Although  the  methodologies  in  these  references  can  be  useful  due  to  their 
simplicity,  no  dynamical  systems  description  is  provided;  hence  the  actual  “pro¬ 
cesses”  do  not  appear,  and  it  is  very  difficult  to  add  dynamical  (say,  queue  and 
packet  rate  dependent)  controls  to  the  formulation.  The  way  that  packet  losses 
affect  individual  sources  and  the  consequent  effects  on  the  full  system  are  not 
modeled  explicitly,  and  it  is  difficult  to  analyze  the  oscillations  or  instabilities 
that  might  be  caused  by  delays.  They  cannot  provide  a  sample-path  or  tran¬ 
sient  analysis.  Models  including  some  of  these  features  appear  in  [2,  5]  under 
simplified  assumptions  on  the  protocol’s  behavior  (e.g.,  an  assumption  in  [2] 
that  loss  probabilities  do  not  depend  on  rates,  or  an  assumption  in  [5]  that  all 
connections  simultaneously  lose  a  packet  when  the  buffer  is  full).  In  order  to 
analyze  more  complex  systems  that  include  buffer  management,  early  marking 
or  discarding,  and  the  impact  of  the  delay  in  the  feedback  loop,  an  alternative 
line  of  research  has  emerged  based  on  fluid  models  using  delay  differential  equa¬ 
tions  methodology;  see  e.g.  [8,  17,  21].  In  [20],  a  fluid  model  of  the  form  of 
a  delayed  ordinary  differential  equation  is  obtained  as  a  limit  of  a  sequence  of 
suitably  scaled  physical  systems,  as  the  number  of  connections  and  the  speed  of 
the  system  grows,  and  where  the  randomness  is  due  to  the  varying  number  of 
non  controlled  connections.  However,  there  is  no  randomness  in  the  limit  model. 
More  detail  on  the  relations  between  [20]  and  our  work  appears  in  Section  7. 

Our  goal  is  to  analyze  heavy  traffic  approximating  models  for  multiplexing 
between  AIMD  and  non  controlled  traffic,  where  the  losses  are  a  consequence  of 
the  actual  underlying  physical  processes,  as  well  as  to  determine  good  controls 
for  buffer  and  loss  management.  The  limit  model  should  retain  the  main  effects 
of  the  randomness  of  the  physical  processes,  which  determines  the  essential 
features  of  the  buffer  and  loss  processes.  The  limit  model  is  not  deterministic, 
but  it  is  much  simpler  to  handle  than  the  original  discrete  stochastic  system, 
and  (as  seen  through  numerical  examples)  it  allows  us  to  get  good  controls  for 
buffer  and  loss  management. 

The  basic  ideas.  As  with  many  models  for  TCP,  we  will  use  a  “stochas¬ 
tic  fluid”  model  for  describing  the  transmission  process;  i.e. ,  rather  than  work 
explicitly  with  the  widow  size;  we  work  with  the  number  of  packets  that  are 
allowed  to  be  sent  per  unit  time.  We  consider  a  model  for  AIMD  traffic  in  the 
operating  region  where  the  system  is  near  capacity.  The  analysis  will  be  “asymp¬ 
totic,”  as  the  system  grows  in  speed.  In  particular,  the  bandwidth  (speed  of 
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the  router)  as  well  as  the  mean  number  of  users  will  be  roughly  proportional  to 
a  parameter  n,  which  is  to  go  to  infinity.  The  analysis  will  be  of  the  so-called 
heavy  traffic  (HT)  type  [13],  which  has  been  of  considerable  help  in  studying 
many  complex  queueing  systems  that  would  be  intractable  otherwise.  Several 
formulations  of  the  demand  process  are  given.  In  all  cases,  there  are  a  certain 
number  of  controlled  users  of  the  order  of  n,  each  having  a  lot  of  data  to  trans¬ 
mit.  These  share  the  channel  with  a  large  and  randomly  varying  number  of 
users  with  smaller  amounts  of  data.  These  are  commonly  referred  to  as  “mice.” 
They  are  in  the  system  for  too  short  a  time  to  be  controlled,  but  might  take  a 
substantial  (40%  or  more)  of  the  total  capacity.  While  each  of  the  mice  (resp., 
each  of  the  controlled  users)  has  identical  statistical  properties,  this  is  only  for 
convenience  in  the  numerical  analysis:  Any  number  of  classes  can  be  handled. 

The  packets  created  by  the  various  users  enter  the  system  in  some  random 
order,  then  are  sent  to  a  buffer  via  various  links,  from  which  they  are  transmit¬ 
ted.  If  the  buffer  capacity  is  exceeded,  then  a  packet  is  said  to  be  “lost.”  Until 
noted  otherwise,  the  round  trip  delay  a  is  the  same  for  all  AIMD  users.  The 
timing  of  the  various  rates  are  as  seen  at  the  buffer  (not  at  the  sources).  They 
depend  on  the  feedback  sent  from  the  buffer  a  units  of  time  ago,  which  reached 
the  source  t\  units  of  time  ago,  was  then  acted  on  and  affected  the  rates  at  the 
input  to  the  buffer  t2  units  of  time  later,  where  t\  +  t2  =  cc. 

We  wish  to  identify  a  region  of  operation  which  is  “near  capacity”  for  large 
n,  and  a  scaling  under  which  the  stochastic  effects  are  apparent.  One  approach 
to  asymptotic  analysis  is  via  a  fluid  model  (e.g.,  [20]).  These  tend  to  average 
or  eliminate  the  effects  of  stochastic  variations  in  the  number  of  users,  mice, 
data  rates,  etc.  But  we  are  more  concerned  with  demonstrating  the  actual 
random  processes  of  losses  and  buffer  content  in  terms  of  the  random  processes 
of  arrivals,  data  levels,  etc. 

We  are  guided  by  the  scaling  used  for  heavy  traffic  models,  as  in  [13].  There 
are  two  related  aspects  to  being  “near  capacity.”  One  is  the  difference  between 
the  mean  packet  creation  rate  and  the  speed  of  the  system,  and  the  other  con¬ 
cerns  the  buffer  size.  Suppose  that  the  total  mean  rate  of  arrival  of  packets  to 
the  buffer  is  vn.  In  order  for  the  system  to  be  in  the  heavy  traffic  regime,  the 
speed  of  exiting  the  buffer  would  have  to  be  slightly  greater  than  vn,  but  not 
so  much  faster  that  the  buffer  is  virtually  empty  almost  all  of  the  time.  If  the 
arrival  process  is  the  superposition  of  many  independent  users,  then  (loosely 
speaking)  the  standard  deviation  of  the  “randomness”  would  be  0(y/n).  This 
suggests  that  if  the  system  is  near  capacity  at  that  time,  then  both  the  buffer 
size  and  the  extra  capacity  would  be  0(y/n).  If  either  the  buffer  or  the  extra 
capacity  are  of  a  larger  order,  then  the  buffer  level  (scaled  by  1/y/n)  would  go 
to  zero  as  n  — »  oo,  and  there  would  be  no  observable  packet  loss.  These  are  the 
usual  orders  in  heavy  traffic  analysis  [13].  The  amplitude  scaling  will  be  1  /y/n. 

The  heavy  traffic  regime  is  one  important  region  of  operation,  one  where 
small  changes  in  the  rates  will  have  major  consequences  for  buffer  overflow 
(i.e,  lost  packets)  and  queueing  delay.  One  can  view  the  system  as  starting 
much  below  capacity,  with  a  lower  packet  rate,  and  with  the  rates  increasing 
until  capacity  is  almost  reached,  at  which  point  the  control  mechanisms  are 
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activated.  Our  analysis  is  confined  to  the  time  that  the  system  is  in  this  heavy 
traffic  regime.  In  the  comments  at  the  end  of  Section  3,  we  will  argue  formally 
that  the  heavy  traffic  regime  is  very  natural,  and  that  a  well  regulated  system 
will  eventually  find  itself  there.  We  will  also  argue  formally  that  one  should  do 
the  control  problem  with  delays  by  allowing  the  controls  to  depend  on  the  path 
over  an  appropriate  time  interval. 

The  controls.  There  are  two  classical  types  of  rate  control  for  each  user.  The 
first  (the  AI  in  AIMD)  is  the  usual  simple  slow  and  steady  linear  increase  in  the 
allowed  rate  of  packet  creation  when  there  are  no  buffer  overflows.  As  noted 
above,  in  the  heavy  traffic  regime,  the  number  of  controlled  users  is  proportional 
to  n  on  the  average,  and  the  excess  capacity  is  0(y/n).  This  suggests  that  the 
cumulative  effect  of  the  first  type  of  control  should  be  a  rate  increase  of  O(y^ri) 
over  all  controlled  users,  which  implies  a  rate  increase  of  0(l/y/n)  per  user.  If 
it  were  larger,  the  system  would  experience  very  serious  packet  losses  in  short 
order.  Thus  we  suppose  that  there  is  a  constant  c  such  that  the  rate  per  source 
increases  by  c/ y/n.  This  is  the  correct  order  in  the  heavy  traffic  regime.  See 
comments  at  the  end  of  Section  3,  where  we  conjecture  that  a  well  regulated 
system  will  eventually  find  itself  in  this  situation.1  It  will  turn  out  that  the 
cumulative  effects  of  this  control  and  of  the  buffer  overflow  controls  are  of  the 
same  order.  The  second  type  of  control  (the  MD  part)  is  the  usual  multiplicative 
decrease  when  there  is  a  lost  packet. 

To  improve  the  performance,  we  also  use  another  type  of  control,  called  a 
preemptive  control ,  by  which  packets  are  selected  at  random  to  be  “marked”  as 
they  enter  the  buffer.  The  chance  of  being  selected  depends  on  the  buffer  state 
and/or  its  input  rate,  and  is  a  control  function  to  be  chosen.  (Early  discarding 
or  marking  has  become  very  popular  since  it  was  proposed  and  deployed  in 
the  well  known  RED  buffer  management  [7,  21].)  The  selection  probability 
will  increase  when  the  system  nears  a  dangerous  operating  point.  There  are 
two  choices  of  how  to  handle  the  marked  packets.  Either  they  are  deleted 
so  that  no  acknowledgment  is  sent,  or  they  are  not  deleted,  but  “modified” 
acknowledgments  are  sent  back  [19].  In  either  case,  the  source  rate  is  decreased 
as  though  the  packet  were  lost.  This  control,  which  anticipates  the  possibility 
of  lost  packets  in  the  near  future,  can  actually  reduce  the  queueing  delay  as 
well  as  the  rate  of  overflow  considerably,  with  minimal  cost  in  lost  throughput; 
numerical  data  will  illustrate  this  point.  In  either  case,  the  use  of  the  preemptive 
control  helps  avoid  oscillations  or  instability  due  to  the  effects  of  bursts  of  lost 
packets  caused  by  the  delays.  Here,  we  work  with  the  second  option,  and  do 
not  delete  the  selected  packets. 

Outline  of  the  paper.  A  general  model  for  the  mice  is  discussed  in  the  next 
section.  Two  properties  are  paramount.  One  concerns  the  asymptotic  (scaled) 
total  number  of  packets  that  have  been  transmitted  by  them  over  any  time  inter¬ 
val.  The  other  concerns  the  current  rate  of  creation  of  packets.  The  assumptions 
are  intuitively  reasonable.  To  emphasize  this,  we  discuss  one  particular  exam- 

1One  could  change  the  model,  using  fewer  sources,  each  with  a  higher  rate,  and  allow  an 
accordingly  faster  increase  in  the  AI  control.  The  analysis  would  be  similar. 
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pie  in  detail,  starting  from  more  “physical”  assumptions.  It  is  supposed  (as  is 
commonly  done)  that  the  mice  enter  with  a  fixed  packet  rate  (possibly  random 
among  the  individuals),  but  that  they  are  in  the  system  for  a  relatively  short 
time,  are  not  controlled  and  do  not  retransmit  lost  packets. 

In  Section  3,  we  consider  the  case  where  there  are  just  n  controlled  users, 
analogously  to  the  setup  in  [20].  Each  of  them  has  a  very  large  (infinite,  here) 
amount  of  data  to  be  sent,  and  is  subject  to  rate  control.  However,  the  ran¬ 
domness  of  the  mice  process  has  a  significant  effect  on  the  total  throughput, 
since  it  is  a  major  cause  of  lost  packets  (buffer  overflows),  and  the  consequent 
rate  control.  The  limit  model  is  a  delayed  stochastic  differential  equation  with 
boundary  reflection.  Note  that  a  delayed  reflection  term  arises.  Section  4  con¬ 
siders  various  extensions  of  the  basic  model  of  Section  2,  including  the  case 
where  there  is  no  buffer  and  where  the  rate  for  the  controlled  users  changes  ran¬ 
domly,  perhaps  due  to  reinitializations;  this  can  be  useful  to  model  a  sequence 
of  TCP  connections  that  are  opened  consecutively  by  the  application  layer,  as 
is  the  case  in  the  HTTP/1.1  version. 

Section  5  deals  with  the  case  where  the  controlled  users  appear  at  random, 
each  with  a  random  amount  of  data  to  be  sent,  and  vanish  when  their  data 
has  been  transmitted.  This  introduces  additional  randomness,  which  (in  the 
asymptotic  limit)  shows  up  via  the  addition  of  new  Wiener  processes  in  the 
dynamics  for  the  rate  process.  Data  that  show  some  of  the  advantages  of  the 
approach  and  how  to  use  it  effectively  are  in  Section  6. 


2  The  Model  for  the  Mice 


Recall  that  we  use  the  name  “mice”  to  describe  any  set  of  sources  whose  trans¬ 
mission  rates  are  uncontrolled  and  with  a  relatively  small  number  of  pack¬ 
ets/source.  Various  cases  where  the  number  of  packets  goes  to  infinity  as  n  — >  oo 
are  covered  by  the  assumptions.  We  suppose  that  the  total  rate  at  which  mice 
packets  are  being  put  into  the  buffer  at  time  t  is  amn  +  y/nl;n(t),  where  am  >  0 
and  £”(■)  is  a  random  process  such  that  f*  £n(s)ds  converges  weakly  to  a  Wiener 
process  with  variance  a‘2m.  More  specifically  (where  =>■  denotes  weak  con¬ 

vergence)  , 


(total  number  of  mice  packets  by  t)  —  namt 
\fn 


C(s)ds 


(2.1a) 


mice  rate(-)  —  nam  £"(•) 


n 

sup  E sup 

n  s<t 


C(r)dr 


<  00, 


“zero”  process, 
each  t  >  0. 


(2.1b) 


(2.1a)  says  that  the  total  mice  packet  rate  is  the  sum  of  a  “fluid”  component 
and  a  part  that  is  essentially  independent  over  short  and  disjoint  intervals.  It 
is  motivated  by  the  central  limit  theorem.  Owing  to  the  complicated  way  that 
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packets  from  different  users  are  scrambled  in  transmission,  it  might  be  hard 
to  say  more,  or  to  specify  the  “mice”  model  more  explicitly.  The  sizes  of  the 
individual  mice  can  grow  with  n,  but  slower  than  O(n).  All  that  we  require  is 
that  (2.1)  hold.  An  interesting  specific  example  of  a  mice  process  is  given  next. 

Example  of  a  “mice”  model.  Consider  the  following  example,  which  was  one 
of  the  motivations  for  the  general  conditions  above.  The  example  is  meant  to  be 
illustrative,  and  does  not  exhaust  the  possibilities.  Suppose  that  the  mice  arrive 
as  a  Poisson  process  with  rate  A mn,  with  each  arrival  having  an  exponentially 
distributed  (and  independent  among  arrivals)  amount  of  packets,  with  mean 
I'm/ l-i-m-  The  packets  are  put  into  the  system  at  a  rate  vm.  The  number  of 
active  mice  at  any  time  is  N^(t),  which  satisfies 


dN™(t)  =  n\mdt  -  fimN^(t)dt  +  dM”(t), 

where  M^(-)  is  a  martingale  with  quadratic  variation  process  f* [nAm+N^ (s) fj,m] ds . 
Let  us  work  with  the  stationary  processes.  Then  N^(-)/n  converges  weakly  to 
the  process  with  constant  values  Xm/ \xm.  The  rate  at  which  mice  packets  arrive 
is  N™(t)vm.  Write  N^(t)  =  nXm/\xm  +  sjnrf^/t).  Then 

dVm(t)  =  -IJ,mVm(t)dt  +  dM™(t)/Vn. 


The  process  M^^/y/n  converges  weakly  to  a  Wiener  process  wm(-)  with  vari¬ 
ance  2Am.  The  process  rj^-)  converges  weakly  to  where  d??m(t)  = 

^l^mVm(t)dt  +  dwm(t).  The  “noise  part”  of  the  arrival  rate  process  for  the  mice 
satisfies 


£”(*) 


^m^m(')  U  V  m  A  m/  /.i  rn 

y/n 


^  VmVm{')  —  £(')' 


Note  that  (2.1b)  holds. 

The  variance  of  (scaled  mice  packet  rate  at  t) / y/n  is,  asymptotically,  u^Am//xm. 
The  (scaled  packet  rate)  correlation  function  is  this  times  For  high  speed 

systems,  both  fim  and  vm  are  large,  while  the  ratio  vm/ \im  (the  mean  number 
of  packets  per  mouse)  is  “moderate.”  In  this  case,  to  show  that  (2.1a)  holds 
“approximately,”  write  (neglecting  the  initial  condition), 


V m(t)  =  [  e  s)dwm(s), 

Jo 


The  dominant  part  is  the  Wiener  process.  Thus,  in  (2.1a),  am  =  vmAm/ \im 
and  the  variance  of  the  Wiener  process  is  cr^  =  2Am[um//xm]2.  The  stationary 
variance  of  the  error  process  (the  last  term  on  the  right)  is  (u^/^)Am//xm.  For 
large  /i m  and  moderate  a the  error  process  is  close  to  the  “zero”  process,  in 
that  it  converges  weakly  to  it  as  jim  — >  oo. 
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We  could  also  suppose,  alternatively,  that  the  individual  mice  send  their 
packets  all  at  once,  but  they  are  interleaved  randomly  with  those  from  other 
sources  along  the  way;  then  we  come  even  closer  to  (2.1). 

3  n  Controlled  Users,  Each  With  Infinite  Back¬ 
log 

In  this  section,  there  are  a  fixed  number,  namely  n,  of  controlled  users,  with 
each  having  a  very  large  (infinite  here,  for  modeling  simplicity)  amount  of  data 
to  be  sent.  Let  Vj(i)  denote  the  rate  for  controlled  source  i  at  time  t,  and 
suppose  that  there  are  positive  a,  such  that  a o  <  r,(0)  <  ai,  so  that  no  single 
source  dominates.  Thus  fQ  Ti(s)ds  is  the  total  number  of  packets  generated  by 
controlled  source  i  by  time  t.  Define  rn(t)  =  E"=i  ri(i)/ni  and  V\  =  f™(0), 
i>2  =  EJr?(0)]2/n>  and  Pn(t)  —  E,:  r*(t)  —  nv i]  /y/n.  Thus  y/npn(t)  is  the  rate 
at  time  t,  centered  about  the  initial  mean  rate  nv\.  The  analysis  commences 
at  the  point  at  which  the  HT  regime  is  entered.2  The  service  rate  (channel 
speed  in  packets  per  second)  is  assumed  to  be  Cn  =  nv i  +  amn  +  b^/n,  b  >  0, 
which  covers  the  mean  requirements  (for  both  persistent  connections  as  well  as 
the  mice  process)  and  gives  an  excess  (over  the  mean  requirements)  of  b^fn. 
The  buffer  size  is  B^/n.  These  are  the  correct  orders  in  HT  analysis  [13].  If 
the  buffer  or  spare  capacity  were  of  a  larger  order,  then  the  number  of  buffer 
overflows,  asymptotically,  would  be  zero. 

When  the  buffer  overflows  (i.e.,  a  packet  is  lost),  that  packet  is  assumed  to 
come  at  random  from  the  various  users,  in  proportion  to  their  individual  current 
rates  of  packet  creation:  The  various  users  (mice  and  controlled)  would  send 
their  packets  in  some  order,  and  the  order  would  be  more  or  less  scrambled  in 
the  course  of  transmission,  so  that  buffer  overflows  can  be  assigned  at  random 
to  the  various  users. 

As  noted  in  the  introduction,  the  standard  multiplicative  decrease  control  is 
activated  by  lost  packets.  I.e.,  there  is  some  constant  n  €  (0, 1)  such  that,  if  the 
dropped  packet  at  time  t  —  a  was  from  connection  i,  then  the  rate  rj(t— )  at  t— 
is  changed  to  r.;(f)  =  (1  —  n)ri{t—). 

The  “preemptive”  control.  The  performance  would  be  improved  if  the 
sources  were  also  signaled  to  reduce  their  rates  as  the  buffer  level  or  total  input 
rate  increases,  but  before  actual  buffer  overflow.  The  type  of  control,  called 
the  preemptive  control,  attempts  to  do  just  this,  analogously  to  what  is  done 
in  the  RED  system.  It  selects  packets  on  arrival,  either  at  random  or  in  some 
deterministic  way  according  to  the  chosen  control  law.  For  notational  simplicity, 
we  suppose  that  the  selection  is  done  randomly.  The  probability  that  a  packet 
entering  the  buffer  at  time  t  is  selected  is  u(t)/yPn,  where  0  <  u(-)  <  umax  <  oo 
is  a  measurable  control  function,  and  is  to  be  selected.  The  selected  packets 
could  be  deleted  as  if  there  was  an  overflow.  A  preferable  alternative,  which 
we  use,  does  not  delete  the  packets,  but  returns  a  modified  acknowledgment, 

2More  on  this  later. 
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which  is  used  to  reduce  the  flow  at  the  source,  similarly  to  what  would  happen 
if  the  packet  were  actually  lost  [19].  Let  Tf  denote  the  minimal  er-algebra  that 
measures  the  systems  data  to  time  t.  Then  u(-)  is  {Ff-  t  <  oo}-adapted.  We 
suppose  that  there  is  a  €  (0, 1)  so  that  if  a  packet  from  source  i  is  selected 
at  time  t  —  a,  then  Ti{t)  =  (1  —  n\ )?y(f— ).  This  preemptive  control  is  to  be 
chosen  by  the  system  designer  and,  when  suitably  selected,  it  can  have  a  major 
beneficial  effect  on  the  overall  operation. 

Buffer  input-output  equations.  We  have 

p"(f)  =  p”( 0)  +  ct  —  [overflow  control  effects]  ,  , 

—  [preemptive  control  effects],  '  ' 

Let  xn  ( t )  denote  1  /  yfn  times  the  number  of  buffered  packets  at  time  t.  Then 

xn(t)  =  xra(0)  +  [(total  input  -  total  output  -  overflow)  by  t ]  j  yfn. 


If  the  buffer  is  not  empty,  then  its  output  rate  is  Cn.  For  modeling  purposes, 
it  is  convenient  to  use  this  output  rate  all  the  time,  even  if  the  buffer  is  empty. 
Then  we  must  correct  for  the  “fictitious”  outputs  when  the  buffer  is  empty.  This 
is  done  by  adding  an  “underflow”  correction  term  Ln(t)  (which  is  the  number 
of  fictitious  outputs  sent  when  the  buffer  is  empty)  as  is  usual  in  heavy  traffic 
analysis  [13].  Let  Un( •))  denote  1  /  yfn  times  the  buffer  overflow.  Now,  using  the 
definition  of  C", /?"(•),  and  the  mice  model  (2.1a),  we  can  write 

xn(t)=xn(0)+  [  [pn{s)-b  +  C(s)}dt-Un(t)+Ln(t).  (3.2) 
Jo 


The  limit  dynamical  equations.  The  following  theorem  gives  the  HT  limits, 
and  identifies  the  limit  control  system.  Define  un(t)  =  f0  un(s)ds. 


Theorem  3.1.  Assume  the  mice  model  (2.1),  that  Cn  =  nv i  +  amn+  yfnb,  and 
that  sup„  |p" (0)|  <  oo.  Then  the  sequence  {xn(-),  p"(-),  u"(-),  Ln(-),  [/"(•)} 

is  tight  in  the  Skorohod  topology.  For  any  weakly  convergent  subsequence,  there 
is  a  process  u(-)  such  that  the  weak  sense  limit  (x(-),  p(-),  u(-),  wm(-),  L(-),  [/(■)) 
satisfies 


dp(t)  =  cdt  —  V2  - - - dU(t  —  a)  +  n\u{t  —  a)dt 

LUl  +  CLm. 


(3.3) 


x(t)  —  :r(0)  =  f  [p(s)  —  b\  ds  +  wm(t)  +  L(t)  —  U (t) ,  (3.4) 

Jo 


where  u(t)  =  f*  u(s)ds.  Let  Tt  denote  the  minimal  a-algebra  that  measures 
(x(s),  p(s),wm(s),u(s  —  a),  L(s),  U(s),  s  <  t).  Then  wm{-)  is  an  IFt-Wiener 
process  with  variance  tr^,  0  <  u(t)  <  umax,  and  u(t)  is  {Tt,t.  <  oo }-adapted. 

Comment  on  the  limit  equations.  Equations  (3.8)  and  (3.9)  are  suggestive 
even  for  more  general  models.  They  capture  much  of  the  essence  of  the  AIMD 
and  the  preemptive  control  mechanisms,  and  retain  the  fundamental  role  of  the 
randomness,  all  for  an  aggregated  and  scaled  system.  Equations  (3.3)  and  (3.4) 


identify  the  correct  limit  control  system.  The  asymptotic  effects  of  the  overflow 
and  preemptive  controls  are  in  the  given  form.  The  control  is  admissible  in 
that  it  is  a  delayed  nonanticipative  (with  respect  to  the  Wiener  process  wm(-)) 
function  satisfying  the  appropriate  bounds. 

Proof.  It  follows  from  the  proof  of  the  reflection  mapping  theorem  in  [13, 
Theorems  3.4.1,  3.5.1]  that  there  is  a  constant  C  such  that,  for  each  0  <  To  < 

T  <  oo, 

(Ln(T)-Ln(T0))+(Un(T)-Un(T0))  <  C  sup  \xn(T0)  +  f  [pn(s)  +  £"(*)]  da 

T0<t<T  L  JT0 

(3.5) 

By  the  assumption  on  pn( 0),  sup„  Esups<t  pn{s)  <  oo  for  each  t.  By  this,  the 
second  line  of  (2.1b),  and  (3.5),  we  have  sup„  EUn(t)  <  oo.  Thus  the  number  of 
buffer  overflows  on  any  bounded  interval  is  0(y/n).  Thus,  since  the  association 
of  overflow  with  source  is  random,  we  can  neglect  the  possibility  that  any  one 
source  will  have  more  than  one  overflow  on  any  finite  interval. 

The  Lipschitz  condition  in  (3.5)  and  the  tightness  criterion  in  [13,  Theorem 
2.5.6]  or  [11,  Theorem  2.7b]  assures  that  the  sequence  {&"(■),  pn(-),  !/"(•),  !/"(•)} 
is  tight  in  the  Skorohod  topology.  The  sequence  {un(-)}  is  obviously  tight  since 
0  <  un(t)  <  umax.  The  fact  that  some  arguments  are  delayed  is  irrelevant. 

We  next  approximate  the  overflow  control  effects  in  (3.1).  Suppose  that 
there  is  a  single  overflow  at  time  t  —  a.  I.e.,  y/ ndUn(t  —  a)  =  1.  Let  J"(t  —  a) 
denote  the  indicator  function  of  the  event  that  the  overflow  is  associated  with 
controlled  source  i.  Then  n(t)  =  r*(t— )(1  —  nl™{t  —  a))  and 

1  n  n 

—  =  -k  V'ri(<-)J”(t  -  a)dUn{t  -  a).  (3.6) 


The  user  with  the  lost  packet  is  selected  at  random,  with  the  probability  that 
controlled  user  i  is  selected  being  (its  rate  divided  by  the  total  rate,  all  at  t  —  a) 


mt-*) 


_ Tj{t  -  a) _ 

Tj{t  -  a)  +  nam  +  ^ nt;n(t  -  a) 


(3.7) 


Use  (3.7)  to  center  (3.6)  about  the  conditional  mean  (given  the  rj(t  —  a),  £,n{t  — 
a),  and  that  dUn(t  —  a)  >  0),  and  rewrite  the  right  hand  term  of  (3.6)  as 


-K^ri(t-) 


r,;(t  —  a)dUn(t  —  a) 


E  ,  rj(t  -  a)  +  nam  +  ^/n£,n(t  -  a) 


(3.8) 


where  M "  ( • )  is  the  martingale 


«  V'u(s-)  [/f(s  -  a)  -  J"(s  -  a)\dUn(s-  a). 


i—1 


By  the  random  association  of  buffer  overflow  to  user,  we  can  show  that 
E\M™(t)\2  =  0(l)Ej2\dUn(s)\2  =  0{l/^)EUn{t). 

S<t 
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Hence  Mn(-)  converges  weakly  to  zero,  and  the  left  side  of  (3.8)  can  be  used  for 
(3.6),  as  n  — »  oo. 

It  was  seen  that  we  can  neglect  the  possibility  that  any  one  source  is  asso¬ 
ciated  with  more  than  one  overflow  on  any  finite  interval.  Thus,  in  evaluating 
the  left  side  of  (3.8),  we  can  suppose  that  rjt  —  a)  =  r»(t— ).  Using  this  and 
(2.1b),  and  dividing  each  part  of  the  term 

_ E lri{t-)ri{t- a) _ 

E j  ri{t  -  «)  +  nam  +  Jn£n(t  -  a) 

by  n,  we  see  that  it  converges  weakly  to  the  constant  process,  with  values 
V2/[vi  +  am],  as  n  — *  oo.  The  above  computations  imply  that,  as  n  — >  oo, 
the  buffer  overflow  control  term  in  (3.1)  is  well  approximatable  by  (kv 2/(71  + 
am\)Un(t  -  a). 

Now,  we  turn  our  attention  to  approximating  the  effects  of  the  preemptive 
control.  Redefine  /"(f)  to  be  the  indicator  of  the  event  that  a  packet  selected  at 
time  t  came  from  controlled  source  i.  Define  Rn(t  —  a)  =  Ej  r*(f  —  a)  +  nam  + 
JnJ(t  —  a),  the  total  (unsealed)  packet  arrival  rate  at  time  t  —  a.  Let  Jn(t) 
denote  the  number  selected  by  t.  Then 

-7=  E  [»•<(*)  -  u(f-)]  =  ^E  -  «)dJ”(t  -  a).  (3.9) 

Jn  Jn 

%  1 

The  mean  rate  at  which  packets  are  selected  at  time  t  —  a  is 

U (*  ~^Rn{t-a).  (3.10) 

Jn 


We  can  model  the  random  selection  times  as  the  jump  times  of  a  jump  process 
with  conditional  jump  rate  un(t)Rn(t)/Jn  at  time  t.  Thus  on  any  finite  interval 
there  are  only  O(Jn)  selections,  and  the  event  that  more  than  one  comes  from 
the  same  source  can  be  neglected.  Thus,  if  a  selection  at  t  —  a  comes  from  source 
i,  we  can  suppose  (without  loss  of  generality)  that  r*(f— )  =  rjt  —  a).  The  rest 
of  the  development  is  similar  to  that  for  the  effects  of  the  overflow  control,  but 
with  </"(•)  replacing  [/"(•).  Thus,  by  centering  J"(f  —  a)  at  its  conditional  mean 
/"(t  —  a),  we  have  the  representation  of  (3.9)  as 

d  s  °><i~f'i(i  -  a> + iM’(t)'  <3ii> 


for  a  martingale  M™ (•).  The  quadratic  variation  of  the  martingale  is  0(1/ Jn), 
hence  it  converges  weakly  to  zero  as  n  — >  00.  Now  centering  d,Jn  about  its 
conditional  mean  yields  the  approximation  to  the  left  hand  term  of  (3.11)  as 


Ki  ri(t—)ri(t  —  a)  un(t  —  a)Rn(t 

Jn  ^  Rn(t)  Jn 

v  %  v  7  v 


a) 


dt  +  dM™  (t) , 


(3.12) 


where  M”(-)  is  a  martingale  whose  quadratic  variation  is  also  0(1/ Jn),  hence 
it  is  asymptotically  negligible. 
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By  what  has  been  said, 


v2ni  /  un(s  —  a)ds 

Jo 

approximates  the  effects  of  the  preemptive  control  for  large  n.  Now,  with  these 
asymptotic  representations  for  the  effects  of  the  controls,  we  see  that  the  limit  of 
any  weakly  convergent  subsequence  of  {xn (•) ,  pn (•) ,  un (•) ,  Un (•) ,  Ln (•) ,  w^-)} , 
satisfies  (3.3)  and  (3.4).  The  Wiener  property  of  wm(-)  is  just  the  assumption 
(2.1).  The  fact  that  it  is  an  {Ft,  t  <  oo}-Wiener  process  is  proved  using  standard 
methods;  for  example  see  [13,  Theorem  6.1.2].  The  limit  u(-)  is  absolutely 
continuous  with  respect  to  Lebesgue  measure,  with  derivative  bounded  by  umax\ 
hence  the  asserted  process  u(-)  exists.  ■ 

Cost  functions  and  nearly  optimal  controls  for  the  physical  system. 

In  order  to  assure  good  performance  of  the  AIMD  connections,  the  buffer  man¬ 
agement  would  implement  control  u(-).  The  quantities  to  penalize  in  the  cost 
are  queueing  delay  (measured  by  ai(-)),  the  loss  of  throughput  due  to  the  control 
(measured  by  — p(-)),  and  buffer  overflow  (measured  by  U(-))3.  Let  us  work  with 
a  discounted  cost  criterion,  where  (3  >  0  can  be  as  small  as  we  wish,  Co  >  0,  and 
the  ki(-)  >  0  are  Lipschitz  continuous: 

/»oo 

W(u)=(3E  /  e~l3t  ([ki(x(t))  —  k2(p(t))\dt  +  codU(t)) .  (3.13) 

Jo 

The  possibility  that  the  ki(-)  are  nonlinear  can  be  useful,  since  (e.g.)  we  might 
wish  to  heavily  penalize  long  queues,  but  not  be  too  concerned  with  short  queues. 

Using  the  methods  of  heavy  traffic  analysis  for  controlled  problems  [13], 
it  can  be  shown  that  the  optimal  costs  for  the  physical  problem  converge  to 
the  optimal  cost  for  the  limit  problem.  If  the  delay  is  zero,  then  the  optimal 
control  for  the  limit  problem  is  of  the  switching  curve  type:  u(x,  p)  takes  the 
maximum  value  on  one  side  of  a  switching  curve  and  is  zero  on  the  other,  and 
the  switching  curve  is  smooth.  The  switching  curve  character  for  a  =  0  follows 
from  a  formal  examination  of  the  Bellman  equation  for  the  optimal  value,  since 
the  control  appears  linearly  in  the  dynamics  and  does  not  appear  in  the  cost. 
The  smoothness  was  implied  by  the  numerical  computations.  See,  for  example, 
Figure  6.1.  Such  switching  optimal  controls  are  nearly  optimal  for  the  physical 
system  for  large  n.  We  note  that  the  cost  (3.13)  is  well  defined,  since  it  can  be 
shown  that  E\p(t)  \  +  EU(t )  <  a\  +  ci2t,  for  some  a*  >  0. 

We  shall  also  consider  an  ergodic  cost  criterion 


7 («)  =  lim  E- 
1  — >00  1 


( ki(x(t ))  -  k2(p(t)))  dt  +  c0U (T) 


(3.14) 


3 Penalizing  buffer  overflow  may  be  important  for  several  reasons.  First,  if  the  mice  cor¬ 
respond  to  real  time  applications,  then  these  applications  will  suffer  due  to  losses.  Secondly, 
the  AIMD  themselves  may  correspond  to  real  time  applications  which  are  ’’TCP  friendly”, 
in  which  case  lost  packets  are  typically  not  retransmitted.  Losses  due  to  overflow  then  again 
degrade  the  quality  of  the  communication. 
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At  present,  there  is  little  theory  concerning  stability  or  ergodicity  theory  for 
delayed  reflected  diffusions  such  as  ((3.3),  (3.4)),  or  ((3.4),  (5.5))  for  the  model 
of  Section  5.  If  the  delay  is  zero  then,  for  any  feedback  control  «(•),  stability  can 
be  shown  and  the  model  ((3.4),  (5.5))  can  be  shown  to  have  a  unique  invariant 
measure;  see,  e.g.,  [13,  Chapter  4].  In  the  numerical  computations  (where  zero 
delay  was  always  used),  we  were  always  able  to  compute  an  optimal  control  for 
the  ergodic  cost  criterion  (with  cost  and  control  well  approximated  by  those  for 
the  discounted  problem  for  small  (3),  and  both  stability  and  convergence  to  the 
stationary  distribution  under  the  optimal  (or  other  reasonable)  controls  were 
apparent. 

Comments  on  the  heavy  traffic  regime.  First,  we  comment  on  the  control 
u(t  —  a)  in  (3.3).  Suppose  that  there  is  a  cost  function  of  either  the  type  (3.13)  or 
(3.14).  Owing  to  the  delay,  the  optimal  control  u(t)  will  not  be  simply  a  function 
of  (x(t),p(t)),  but  rather  a  function  of  the  path  segment  {cc(s),  p(s),  t  —  a  < 
s  <  t}  [9,  22].  Although  there  is  some  progress  with  numerical  methods  for 
computing  and  approximating  optimal  «.(•)  when  a  >  0,  good  algorithms  are 
not  yet  available.  Controls  u(t )  that  depend  just  on  the  value  of  (x(t),p(t)) 
are  subject  to  oscillations.  Those  that  depend  as  well  on  the  recent  past  can 
avoid  oscillations,  since  they  “remember”  recent  control  values  and  can  select 
the  current  value  accordingly.  It  is  likely  that  using  the  full  potential  of  the 
delay  dynamical  system  can  improve  the  operation  and  keep  the  system  in  the 
heavy  traffic  regime.  Following  are  some  conjectures.  They  are  reasonable,  but 
unproved  at  this  time. 

Suppose  that  the  system  starts  far  from  the  heavy  traffic  regime,  and  that 
the  “slow”  increase  in  packet  rate  is  initially  c  for  each  of  the  n  users.  Then  the 
total  increase  in  the  rate  for  the  persistent  connections  is  nc.  Capacity  will  be 
reached  quickly.  One  can  try  to  pose  a  control  problem  with  another  preemptive 
control,  where  losses  are  penalized  heavily.  One  expects  that  (roughly  speaking) 
at  a  time  t  when  the  average  rate  reaches  a  level  where  approximately  a  units 
of  time  later,  it  will  be  within  0(^/n)  of  capacity,  this  control  starts  to  act, 
and  selects  packets  for  the  modified  acknowledgment.  The  packet  rate  into 
the  buffer  will  keep  increasing  until  the  rate  reducing  effects  of  the  feedback 
reach  the  buffer  a  units  of  time  later.  After  time  t,  since  the  control  “knows” 
the  recent  state  values,  it  knows  how  many  packets  it  has  already  selected,  and 
adjusts  new  selections  accordingly.  The  y7 n  level  is  used  because  that  represents 
the  effects  of  the  randomness.  The  control  will  start  to  act  when  asserted  since 
otherwise  there  will  be  large  losses.  It  is  reasonable  to  expect  that  such  behavior 
would  avoid  oscillations  and  bring  the  system  to  the  heavy  traffic  regime,  where 
an  additional  fine  control  can  be  exerted.  Similar  comments  apply  to  the  case 
where  the  delay  depends  on  the  user.  We  conjecture  that  a  fuller  development 
and  exploitation  of  control  theory  when  the  controls  are  delayed  will  have  a 
major  impact. 
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4  Extensions  of  the  Model  of  Section  3 


No  Buffer.  Suppose  that  there  is  no  buffer,  so  that  if  the  total  current  packet 
rate  exceeds  the  channel  speed,  then  the  excess  packets  are  rejected.  The  forms 
of  the  input  processes  and  channel  speed  (service  rate)  are  as  in  the  last  section, 
but  in  lieu  of  (2.1),  we  assume  that  £"(■)  converges  weakly  to  a  process  £(•),  as 
in  the  example  in  Section  2.  Since  there  is  no  buffer  to  overflow,  the  “reject” 
process  Un{-)  needs  to  be  defined.  Define 


yn(t )  = 


cn- 


'n(t)  +  amn+  VnC(i) 


/va=[b-pn(t)-c(t)]. 


(d.i) 

the  scaled  difference  between  the  channel  speed  and  input  packet  rate  at  t.  Then 
the  scaled  number  of  rejected  packets  is 


Un{t)  =  f  ds  =  [  r(s)  +  Pn(s)  -  b}+  ds  (4.2) 

Jo  Jo 

Suppose  that  the  correlation  time  of  £"(•)  is  short  (e.g.,  large  pm  in  the 
special  mice  model)  of  Section  2.  Then  a  law  of  large  numbers  argument  can  be 
used  to  show  that  £"(t)  can  be  “integrated  out”  of  (4.2),  in  that,  as  n  —>  oo  and 
the  correlation  time  goes  to  zero,  the  integrand  can  be  replaced  by  the  average 
over  £"(t).  This  simplifies  the  expression  for  Un(-),  and  the  limit  equation  for 
the  scaled  and  centered  rate  process  pn(-)  is 


p(t)  =  cdt  —  V2 


liq  +  a7 


-U(t  —  a)  +  K\u(t  —  a)dt 


(4,3) 


where  U{t)  =  Ep  [£(t)  +  p(t)  —  6]  +  ,  and  the  expectation  is  over  the  £(f).  Then 
we  have  a  deterministic  limit,  which  is  not  a  priori  obvious.  The  randomness 
due  to  the  mice  in  the  arrival  process  does  not  appear  explicitly  in  (4.3),  but  it 
affects  the  value  of  the  expectation  that  yields  the  overflow  rate  U (t) . 

Let  us  look  a  little  more  closely  at  the  example  in  Section  2.  As  pm  — >  oo, 
we  would  also  have  that  vm  — >  oo,  to  keep  the  total  mean  data  per  mouse  from 
going  to  zero.  If  we  suppose  that  Xm{vm/ pm)  is  bounded,  then  the  stationary 
variance  of  £(■)  would  be  0(vm),  which  implies  that  the  excess  capacity  factor 
b  would  have  to  be  0(y/u^),  if  large  loss  rates  are  to  be  avoided.  This  is 
useful  scaling  information.  It  is  hardly  surprising,  since  we  no  longer  have  the 
buffer  to  “integrate”  the  mice  process,  and  we  must  deal  directly  with  the  large 
variations  in  the  £"(•)  instantaneous  rate  process.  In  order  to  avoid  huge  losses, 
the  excess  capacity  must  be  some  large  constant  times  the  standard  deviation 
of  this  process. 

Finally,  we  note  that  the  value  of  an  optimal  control  at  time  t  for  the  limit 
process  (which  will  be  applied  at  time  t  +  a)  need  only  depend  on  p(s),t  —  a  < 
s  <  t.  Since  such  controls  are  nearly  optimal  for  the  physical  process  in  heavy 
traffic,  we  see  that  an  nearly  optimal  control  can  depend  only  the  rate  pn(-)  on 
[i  —  a,  t\,  and  not  on  the  more  rapidly  changing  “mice”  process  £”(•)• 
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Analogous  results  holds  for  the  model  of  Section  5,  but  there  the  randomness 
due  to  the  arrival  and  departure  processes  of  the  controlled  users  remains  in 
the  limit.  As  for  the  case  of  concern  above,  only  the  £n(f)  process  would  be 
“integrated  out.” 

Random  ri(0).  In  the  rest  of  this  section,  we  suppose  that  there  is  a  buffer, 
as  in  the  problem  of  Section  3.  Suppose  that  the  initial  values  of  the  rates  are 
random,  identically  distributed,  and  mutually  independent,  with  Pr,;( 0)  =  Vi 
and  E[ri( 0)]2  =  v2 ■  Then  all  the  asymptotic  results  continue  to  hold. 

Randomly  changing  rates.  In  some  internet  applications,  where  a  user  sends 
a  sequence  of  consecutive  TCP  connections,  the  rate  of  transmission  is  reinitial¬ 
ized  for  each  new  TCP  transfer  (e.g.  HTTP/1.1).  We  next  propose  a  model  of 
which  this  scenario  is  a  special  case.  Suppose  that  the  users  change  the  packet 
transmission  rates  at  random,  and  each  with  rate  Ao-  The  new  rates  (which 
are  uniformly  bounded)  are  chosen  randomly  with  the  same  first  two  moments. 
More  precisely,  there  are  mutually  independent  Poisson  processes  P,  ( • )  all  with 
rate  Ao-  When  P,(-)  jumps,  the  rate  for  user  i  is  replaced.  The  set  of  replace¬ 
ments,  over  all  users  and  time,  is  mutually  independent,  and  independent  of 
all  other  “driving”  processes.  Let  q  denote  the  canonical  rate  replacement,  and 
define  v\  =  Eq,  v2  =  Eq2,  v2  =  E[q  -  tq]2  =  v2  -  vf,  and  Qn(t)  =  Y<iri(i)- 
Then 


dQn(t)  =  y/ncdt  —  [effects  of  controls]  —  Aq  [Qn(t)  —  nv i]  dt  +  dM™{t),  (4.3) 


where  the  martingale  M”(-)  can  be  shown  to  have  quadratic  variation  process 


A0  [  ^  E  [ri(s)  -  q}2  ds  =  X0n  f 
Jo  i  Jo 


~Ei  rf(s) 
n 


g/h  E,  rj(s) 
n 


+  v2 


ds , 


(4.4) 


where  the  expectation  is  over  q  only.  Recall  the  definition  pn(t)  =  [ Qn(t )  — 
nvi]/y/n.  It  can  be  shown  that  pn(-)  is  tight  and  that  the  limit  of  any  weakly 
convergent  subsequence  satisfies 


dp(t)  =  cdt  —  v2 


li>i  +  a, 


-dU(t  —  a)  +  Kru(t  —  a)dt 


Ao p(t)dt  +  dwr(t),  (4.5) 


where  the  Wiener  process  wr(-)  has  variance  2Aor>2-  The  limit  system  equations 
are  (3.4)  and  (4.5).  The  control  is  as  in  Theorem  3.1. 

Delay  depending  on  the  user.  Up  to  now,  all  users  had  the  same  delay.  The 
general  theory  can  handle  user-dependent  delays.  Suppose  that  user  i  has  delay 
oti  <  D  <  oo.  Let  the  buffer  overflow  at  time  s  with  a  packet  from  user  i.  The 
information  will  reach  user  i  at  time  s  +  t\^.  Thus,  at  time  f,  user  i  receives 
information  concerning  overflows  at  time  t  —  Uy,  and  its  response  reaches  the 
buffer  f2,i  units  of  time  later,  with  +  t2 y  =  a,.  This  leads  to  dUn(t  —  a)  in 
the  1th  summand  in  (3.8)  being  replaced  by  dUn(t  —  a,;).  To  simplify  matters  in 
this  brief  presentation,  first  suppose  that  all  initial  rates  are  equal:  r.;(0)  =  V\. 
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Then,  for  large  n,  the  main  term  in  (3.8)  is  approximately 


n(t  -  ai)dUn(t  -  a,;) 


i=1  r3  (t  -  ai )  +  nam  +  y/n£n(t  -  Oti ) 

KV\  1 


CLm  +ViHZ 


-Y,dUn(t-aj). 


i- 1 


More  succinctly,  with  /?”(•)  being  a  measure  with  mass  1/n  at  at,  write 


v2  n  rD 

V  dUn(t  -  aA  =  v\  /  dUn(t.  -  a)l3n(da). 

n  i~i  Jo 


Suppose  that  the  distribution  of  delays  /?"(■)  converges  weakly  to  a  distribution 
/?(•).  Then  the  dU(t  —  a)  in  (3.3)  is  replaced  by  / dU(t  —  a) (3 {da).  All  else 
remains  the  same.  If  the  r.;(0)  are  not  all  equal,  then  redefine  /3n(-)  to  have 
weight  rf(0)/n  at  a*,  suppose  that  /?"(•)  =»  /?(•),  and  replace  the  right  side  of 
the  last  expression  by  dU{t  —  a)/3{da).  Details  of  the  proof  are  omitted. 


5  A  Stochastic  Process  of  Finite  AIMD  Connec¬ 
tions 

In  the  model  of  Section  3,  the  number  of  users  is  fixed  at  n.  Now,  we  consider  a 
model  where  the  controlled  users  arrive  independently  and  randomly  and  leave 
at  random,  with  the  arrival  process  independent  of  the  mice  process.  New  users 
come  from  an  unlimited  population,  with  (Poisson)  arrival  rate  An.  Each  new 
user  comes  with  an  exponentially  distributed  number  of  data  packets,  each  with 
mean  V\j and  independent  of  the  mice  process  and  arrival  times.4 

With  this  model,  as  with  the  previous  ones,  the  buffer  overflows  (i.e.,  packet 
losses)  are  created  by  the  physical  process  and  not  imposed.  Note  that  the  mean 
amount  of  data  in  a  new  source  does  not  depend  on  n.  The  parameter  n  scales 

4 Exponential  distribution  of  interarrival  times  and  session  duration  are  more  appropriate 
for  telephone  calls  than  for  data  connections.  Thus  this  model  is  expected  to  be  more  useful  for 
VoIP  applications  that  use  TCP  friendly  mechanisms  to  regulate  their  rate.  The  “exponential” 
assumptions  can  be  helpful  even  for  the  data  connections  for  some  preliminary  dimensioning 
purposes. 

Non  exponential  distributions  can  be  handled  as  well,  with  an  increase  in  the  dimensionality 
of  the  limit  model.  For  example,  a  fc-stage  Erlang  model  would  require  a  ^-dimensional  process 
to  represent  the  rate  process.  The  mathematical  development  and  results  are  similar.  This 
higher  dimensionality  is  a  handicap  for  numerical  computations,  say  via  the  Markov  chain 
approximation  method  [12],  or  a  pathwise  approximation  method.  But  it  is  not  a  serious 
handicap  for  simulation.  Indeed,  simulating  the  approximating  limit  model  is  substantially 
simpler  than  simulating  the  physical  process,  when  there  are  very  many  users. 

Experimentation  with  the  basic  model  can  lead  to  insights  that  are  useful  for  more  general 
cases.  For  example,  numerical  results  for  the  basic  model  with  no  delay  indicate  that  threshold 
controls,  based  on  the  rate  only,  provide  good  approximations  to  the  values  obtained  by 
optimal  controls.  This  observation  provides  a  basis  for  getting  good  controls,  which  would  be 
very  hard  to  compute  otherwise,  for  more  general  large  size  systems. 
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the  system  speed  and  mean  number  of  users  only.5  The  source  (i.e.,  the  user) 
stays  “active”  until  all  data  is  sent,  and  then  disappears.  Time  is  still  measured 
at  the  buffer  and  the  mice  model  is  (2.1).  For  simplicity,  suppose  that  the  initial 
rate  of  each  new  controlled  source  is  V\. 

First  suppose  that  there  are  no  controls  (constant  transmission  rate  from 
each  source)  and  buffer  overflows  are  not  retransmitted.  Then  the  packets  are 
sent  from  each  active  source  to  the  buffer  at  a  rate  v\.  The  mean  time  that  a 
source  is  active  is  1/p.,  and  the  total  rate  at  which  the  sources  drop  out  at  t  is 
piV"(i ),  where  Nn(t )  denotes  the  number  of  active  sources.  The  (stationary) 
mean  number  of  sources  in  the  system  is  nA/p.  Hence,  the  analog  of  the  channel 
speed  Cn  of  Section  3  is  Cn  =  Vin[X/ fj]  +  amn+by/n,  where,  again,  b^/n  denotes 
the  excess  capacity  over  the  mean  rate  n[v iA/p  +  am\.  On  departure  of  a  user, 
its  rate  v\  is  lost.6  We  suppose  that  1/p  is  large  enough  relative  to  the  delay  a 
so  that  there  is  enough  time  for  many  round  trips. 

Now  suppose  that  the  input  rates  from  the  non-mice  sources  are  actually 
controlled.  There  are  several  approaches  that  one  can  take  for  the  source  de¬ 
parture  process.  One  approach  supposes  that  the  departure  rate  (of  an  AIMD 
connection)  is  p,  and  does  not  depend  on  the  current  packet  transmission  rate 
for  the  source.  Then  the  lost  packet  rate  if  connection  i  leaves  is  ?y(i).  This  sit¬ 
uation  arises  when  the  AIMD  connections  correspond  to  real  time  applications 
that  have  a  dynamic  compression  rate  (which  is  then  ’’TCP  friendly”).  In  these 
applications,  lost  packets  are  not  retransmitted  (the  possibility  of  lost  packets 
might  be  anticipated  in  the  coding).  For  simplicity  in  the  development,  this  is 
the  approach  that  will  be  taken.  ' 

The  dynamics  and  limit  for  the  rate  process.  The  details  are  similar 
to  those  in  Section  3,  except  for  the  treatment  of  the  randomness  due  to  the 
arrivals  and  departures  for  the  controlled  users,  and  we  will  concentrate  on  this 
point.  Write  Nn(t)  =  nN  +  v/ni/rl(t),  N  =  A/p.  Since  the  user  arrival  process 
is  Poisson  and  the  departure  rate  per  user  is  constant, 

dNn(t)  =  Xndt  -  nNn{t)dt  +  dM£(t)  -  dAfJ(i).  (5.1) 

Here  M™(-)  is  the  martingale  associated  with  the  arrival  process  and  has  quadratic 
variation  process  nXt,  and  M%(-)  is  the  martingale  associated  with  the  depar¬ 
ture  process  and  has  quadratic  variation  process  p  f0  Nn(s)ds.  For  simplicity, 
suppose  that  Nn(-)  is  stationary.  It  follows  from  this,  (5.1),  and  the  cited  values 

5The  rate  of  arrivals  of  new  users  can  be  a  smaller  order  of  ?i,  and  then  they  would  each 
have  an  amount  of  data  that  would  depend  on  n.  E.g.,  rate  of  arrival  0(y/n),  with  data 
0(y/n).  In  this  case  the  rate  of  work  on  each  source  is  0(y/n ),  so  that  the  average  sojourn  in 
the  system  is  still  0(1). 

6  Strictly  speaking  a  source  should  not  depart  until  an  acknowledgment  of  its  last  transmis¬ 
sion  has  been  received.  But  our  approximation  to  the  actual  departure  rule  has  little  effect, 
since  the  order  of  lost  packets  is  still  0(y/n),  and  fi  is  large. 

7An  alternative  approach  replaces  the  value  of  ^  by  a  time  varying  quantity  to  reflect 
the  fact  that  even  if  the  service  rate  per  source  changes  the  total  amount  of  data  per  source 
doesn’t.  For  example,  if  the  allowed  data  rate  for  an  AIMD  connection  is  cut  in  half  due  to 
an  increase  in  the  number  of  sources,  then  the  value  of  the  connection  departure  rate  for  that 
source  should  be  cut  in  half.  The  mathematical  development  of  this  situation  is  much  harder. 
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of  the  quadratic  variations,  that  the  sequence  Nn(-)/n  converges  weakly  to  a 
process  with  constant  value  N  =  X/p,  as  n  — >  oo.  Also,  vn{-)  satisfies 

dvn(t)  =  —pvn(t)dt  +  [dM”(t)  —  (5.2) 

The  quadratic  variation  of  the  scaled  martingale  term  in  (5.2)  is  A t+fi  /Q*  Nn(s)ds/n, 
which  converges  weakly  to  2A t.  The  sequence  z/"(-)  converges  weakly  to  v(-), 
where 

dv(t)  =  — / iv(t)dt  +  dw(t ),  (5-3) 

where  'Uj(-)  is  a  Wiener  process  with  variance  2A. 

Returning  to  the  rate  process,  write  JTrj(t)  =  Qn(t)  =  nR  +  V™Pn(t)’ 
where  R  =  v±X /  p.  The  process  Qn(-)  satisfies 

dQn(t)  =  Avi  ndt  —  pQn{t)dt  +  cy/ndt 

—  [effects  of  overflow  and  preemptive  controls]  +  VidM™(t)  —  dM()l1(t), 

(5.4) 

where  M ^(O  is  the  martingale  associated  with  the  “rate  departure”  process  and 

it  has  quadratic  variation  process  n  f*  ri(s)^s-  This,  divided  by  n,  converges 
weakly  to  the  process  with  values  vfXt,  as  n  — >  oo.  Finally,  following  the 
procedure  used  in  the  proof  of  Theorem  3.1,  it  is  not  hard  to  show  that  pn( •)  => 
p(-),  where 


dp(t)  =  —pp(t)dt  +  [A /p]cdt - dU(f  ~  a )  (r  ^ 

vl[X/P\  +  CLm  (5-5) 

+v\k\  [A  /  p]u(t  —  a)dt  +  V\dw. 

Approximations  to  the  optimal  via  the  limit  model.  The  limit  system 
equations  are  (3.4)  and  (5.5).  The  comments  made  after  Theorem  3.1  concerning 
the  convergence  of  the  optimal  costs  for  the  physical  problem  to  that  for  the 
limit  also  hold  here. 


6  Numerical  Data:  Optimal  Preemptive  Con¬ 
trols 

It  is  not  possible  at  present  to  compute  optimal  policies  when  there  is  a  delay 
in  the  control  (although  there  is  promising  work  being  done  on  the  development 
of  numerical  algorithms),  so  we  set  a  =  0.  The  results  still  shed  light  on  the 
system  behavior  when  the  delay  is  small  relative  to  the  time  constant  in  (3.3) 
or  (5.5). 

Numerical  results  were  obtained  for  the  optimal  control  and  costs  for  the 
model  of  Section  5  with  the  cost  function  being  either  (3.13)  or  (3.14),  with 
ki(x )  =  CiX,  k'iip)  =  C2P-  The  results  for  the  two  cost  functions  were  nearly 
the  same  when  (3  <  .02,  and  the  ergodic  case  will  be  described.  The  numerical 
method  was  the  Markov  chain  approximation  method  [12],  which  is  the  most 
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Switching  curve  in  (x.rho),  unit  =  .1 ,  rho  =0  is  at  128. 


Figure  1:  A  switching  curve;  no  delay. 


versatile  current  approach  for  controlled  reflected  diffusions.  Only  a  few  details 
can  be  given  here.  Use  c  =  l,A//z  =  4 ,b  =  l,vi  =  1.5 ,am  =  4, k  =  =  .5, 

cr^  =  4.  Since  there  is  no  delay  the  control  is  a  function  of  (p(t),x(t)).  We 
used  the  bound  0  <  u(x,p)  <  1.  The  buffer  capacity  is  12.8 ^Jn  packets,  and 
Co  =  100,  Ci  =  1,C2  =  5,  reflecting  our  desire  to  penalize  lost  packets  most 
heavily.  The  mice  account  for  about  40%  of  the  traffic  and  the  system  is  quite 
“noisy,”  since  the  variances  of  the  Wiener  processes  driving  (x,p)  are  (9,4.5). 

The  optimal  preemptive  controls  are  determined  by  a  switching  curve:  u(x,  p) 
0  below  the  curve  and  equals  its  maximum  value  above  the  curve.  The  curve 
obtained  for  our  example  in  the  asymptotic  regime  is  given  in  Figure  6.1.  As 
we  see,  in  (a:,  p)  space,  the  curve  is  initially  (for  small  a:)  almost  a  straight 
line  with  a  slightly  decreasing  slope  as  x  increases.  As  the  buffer  fills  up,  the 
slope  becomes  sharply  more  negative,  as  expected.  The  optimal  cost  for  the 
problem  with  preemptive  control  was  about  l/10th  of  that  without.  In  general, 
The  values  of  the  cost  components  (stationary  mean  values  of  a :(t),p(t),  and 
lirn^oo  EU(t)/t)  are  more  significant  than  the  optimal  cost,  since  they  give  us 
information  on  the  tradeoffs.  Optimal  control  is  not  of  interest  for  its  own  sake, 
but  rather  for  the  information  provided  on  good  design,  and  tradeoffs  among 
the  cost  components  as  the  weights  change. 


For  the  uncontrolled  problem,  the  sum  of  the  buffer  overflow  rate  for  all  users 
was  5.35 y/n,  vs.  0.28 y/n  under  the  optimal  control  for  the  given  cost  coefficients. 
The  mean  queue  was  virtually  full  for  the  uncontrolled  case,  compared  to  an 
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average  of  one-third  full  under  optimal  control.  The  total  input  rate  for  the 
controlled  users  was  reduced  by  an  average  of  0.36\/n  under  optimal  preemptive 
control,  compared  with  an  increase  of  6.3 y/n  with  no  control.  Thus  to  get  an 
improvement  in  overflow  of  about  20  times  cost  a  fractional  reduction  in  the 
throughput  of  (6.3  +  0.36)/[(viA//ii)  +  am]^/n)  =  0.666 /y/n. 

If  the  buffer  size  is  increased,  its  average  percentage  occupancy  is  about  the 
same  (queue  size  is  not  weighted  heavily),  the  average  Ep  increases,  and  the 
average  overflow  rate  does  not  change  dramatically  (e.g.,  doubling  the  buffer 
only  halves  the  overflow,  under  our  parameters).  The  optimal  system  adapts 
to  an  increased  buffer  size  mainly  by  increasing  the  average  flow,  keeping  the 
queue  size  roughly  in  proportion  to  the  buffer  size,  an  interesting  fact  in  itself. 
Of  course,  a  larger  weight  on  x  will  reduce  the  average  queue  size. 

These  numbers  illustrate  the  type  of  tradeoffs  that  are  possible.  One  pays 
for  reduced  overflow  by  reduced  packet  rate.  But  the  packet  rate  is  reduced  only 
where  it  does  the  most  good.  The  tradeoffs  vary  with  the  cost  coefficients.  To 
use  the  method  effectively,  one  makes  a  series  of  runs,  varying  the  coefficients 
c*  This  yields  a  set  of  possible  tradeoffs  between  the  competing  criteria.  In 
each  case,  the  tradeoff  is  under  an  optimal  control.  The  approach  to  the  use  of 
numerical  methods  and  heavy  traffic  approximations  is  similar  to  what  was  done 
for  the  problem  of  input  control  of  a  multiplexer  system  in  [14].  A  comparison 
with  threshold  controls  shows  that  the  effects  of  the  optimal  control  can  be  well 
approximated  by  a  threshold  control  depending  on  p  only,  for  appropriate  values 
of  the  threshold.  The  cost  components  for  the  no  control,  optimal,  and  threshold 
cases  are  summarized  in  Table  1.  If  the  threshold  controls  are  activated  only 
when  the  buffer  exceeds  some  modest  level,  their  performance  is  even  better. 
Keep  in  mind  that  the  described  optimal  control  and  costs  are  for  a  very  heavy 
weight  on  overflow. 


Table  1.  Cost  components. 

under  run  type 

buf  overflow/yTI 

Ex 

Ep 

no  cont. 

5.35 

11.92 

6.35 

opt.  cont. 

.28 

4.4 

-.36 

thresh  p  =  0 

.69 

7.6 

1.46 

thresh  p  =  —  1 

.48 

6.4 

.98 

thresh  p  =  —  3 

.33 

4.9 

.2 

7  Appendix:  Comparison  With  a  Fluid  Model 

Reference  [20]  also  concerned  a  limit  approximation  for  large  systems  and  justi¬ 
fied  the  use  of  a  delayed  deterministic  differential  equation  as  an  approximation 
for  a  certain  class  of  problems.  Since  there  are  major  differences  between  that 
work  and  this,  apart  from  the  different  scaling,  and  since  that  paper  is  the  main 
other  current  work  on  the  use  of  limit-delay  equations  for  AIMD  models,  a  brief 
discussion  of  some  of  the  differences  is  worthwhile. 
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In  the  basic  model  of  Section  3,  capacity  (i.e.,  bandwidth)  scales  linearly  with 
n,  and  so  does  the  number  of  sources.  The  packet  rate  for  each  source  is  0(1). 
Our  general  approach  also  allows  the  possibility  that  the  number  of  sources 
grows  more  slowly  with  n,  with  the  packet  rate  per  source  growing  accordingly 
faster.  While  there  are  no  explicit  capacity  constraints  in  [20],  it  is  clear  that 
the  bandwidth  (BW)  is  proportional  to  their  n2,  and  we  use  this  fact  below. 
They  use  a  fixed  number  of  connections  of  the  order  of  VBW  (and  no  analog 
of  the  models  of  Sections  4  and  5),  each  sending  packets  at  rate  0(\/BW).  The 
number  of  mice  connections  grows  linearly  with  VBW,  and  so  does  the  rate  of 
each  mouse.  Time  is  divided  into  “decision  intervals”  of  length  0( l/\/BW),  and 
the  rates  are  (perhaps  unrealistically)  averaged  over  these  successive  intervals 
before  feedback  and  decisions.  This  averaging  over  0(v/BW)  packets  before 
feedback  effectively  eliminates  the  randomness  due  to  the  mice.  We  work  closer 
to  system  capacity  where  the  effects  of  random  variations  are  greater,  and  it 
is  the  true  instantaneous  randomness  that  causes  the  losses  and  activates  the 
controls. 

The  total  overall  rates  of  increase  of  the  packet  rate  due  to  the  slow  additive 
control  is  the  same  here  and  in  [20].  In  [20]  the  “slow  constant  rate  of  increase 
of  the  packet  rate”  of  each  connection  in  the  nth  model  (the  one  corresponding 
to  n  TCP  connections)  increases  by  1/n  per  each  time  slot,  so  that  in  terms 
of  real  time  the  total  rate  of  increase  does  not  depend  on  n.  Thus  the  total 
rate  of  increase  is  of  the  order  of  \/BW,  as  in  our  case.  In  our  model,  the 
packet  loss  of  each  AIMD  source  is  random  and  determined  by  the  loss  process 
associated  with  that  source.  This  is  in  conformance  with  the  objectives  of  buffer 
management  schemes  [7].  In  [20],  in  contrast,  all  AIMD  sources  have  the  same 
instantaneous  dynamics,  hence  identical  losses.  An  important  advantage  of  the 
work  in  [20]  is  that  the  model,  being  deterministic,  is  much  simpler.  Hence, 
under  its  assumptions,  one  can  more  conveniently  explore  some  of  the  effects  of 
delays. 
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